feat(cli): integrate Obscura headless browser (required, Apache-2.0)#155
Merged
feat(cli): integrate Obscura headless browser (required, Apache-2.0)#155
Conversation
Add Obscura (h4ckf0r0day/obscura) as a required CLI alongside shot-scraper: - shot-scraper keeps screenshots, PDF, accessibility tree, auth interactive - Obscura takes JS eval, multi-URL parallel scraping, CDP server scenarios - Rule: visual → shot-scraper, data → Obscura Bench (per upstream): 30 MB RAM (vs 200+ MB Chrome), 70 MB binary (vs 300+ MB), 85 ms page load (vs ~500 ms). V8 + Chrome DevTools Protocol, Puppeteer/Playwright compatible, stealth mode anti-fingerprinting, native Rust parallel workers. Changes: - framework/tools/cli-registry.json: new entry priority=required, category=browser - framework/cli/internal/installer/catalog.go: --with-obscura ExternalModule (binary download per platform, cargo install fallback) - framework/agents/_shared/obscura-protocol.md: shared protocol with decision matrix vs shot-scraper, install, command reference, fallback rule - CLAUDE.md, .claude/rules/install-reference.md: document Obscura adoption - 4 web-capture agents extended with obscura-protocol and shot-scraper javascript calls converted to obscura fetch --eval: - frontend/03-visual-auditor.md (10 call sites) - frontend/02-frontend-qa.md - audit/32-seo-auditor.md - docs/17-agamotto.md (Penpot extraction) https://claude.ai/code/session_015HAvQLiLeNyP3LUURaEyeC
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (23)
📝 WalkthroughWalkthroughThis PR introduces Obscura, a Rust-based headless browser engine, as a complementary tool to shot-scraper. It establishes a shared protocol defining when agents should use each tool, registers Obscura in the installer and tool registry, extends multiple agent workflows to adopt Obscura for JS evaluation and parallel scraping, and updates test expectations for agent model variants. ChangesObscura CLI Integration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.1)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…t policy Pre-existing test debt surfaced on PR #155 CI: 1. 7 golden files had expect_model: sonnet but the agents were retargeted in commit 1d4f351 ("align model selection across 87 agents"): - agathe, benjamin, frontend-orchestrateur → opus - gandalf, godspeed, tools-checker → haiku - alex → sonnet (was previously expected as opus) 2. framework/tests/cheatheet.test.mjs:73 had validModels = ['opus', 'sonnet'], missing 'haiku' which was introduced for godspeed/gandalf/tools-checker. Both test suites now pass (610 golden + 30 cheatheet asserts). The Go CLI coverage gate (70%) is left untouched — pre-existing on this branch (38%) and on main (55%); fixing it requires substantial test additions out of scope for the Obscura PR. https://claude.ai/code/session_015HAvQLiLeNyP3LUURaEyeC
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add Obscura (h4ckf0r0day/obscura, Apache-2.0, Rust) as a
requiredCLI alongsideshot-scraper, with a clear split:Per upstream, Obscura is a lightweight headless browser (V8 + CDP, Puppeteer/Playwright compatible, stealth mode): 30 MB RAM (vs 200+ MB Chrome), 70 MB binary (vs 300+ MB), 85 ms page load (vs ~500 ms). Native Rust workers make
obscura scrape~5–10× faster thanshot-scraper multion batches > 20 URLs.Changes
framework/tools/cli-registry.json— new entry,priority: required,category: browser, install per-OS (binary download) +cargofallback, examples forfetch/scrape/serve.framework/cli/internal/installer/catalog.go—--with-obscuraExternalModule with platform-aware install script (uname -s/uname -m→ arm64-macos / x86_64-macos / x86_64-linux), cargo fallback for unsupported platforms. Go build + tests pass.framework/agents/_shared/obscura-protocol.md— new shared protocol: install, decision matrix vs shot-scraper, command reference (fetch/scrape/serve), ulk patterns (SEO, alt audit, link check, sitemap crawl), limitations, fallback rule.CLAUDE.md,.claude/rules/install-reference.md— document Obscura adoption (2026-05-07),--with-obscuraflag, install methods, the shot-scraper coexistence rule.obscura-protocoland JS eval calls converted fromshot-scraper javascript "$URL" "..."toobscura fetch "$URL" --eval "..."(10 sites in visual-auditor, 1 in frontend-qa, 1 in seo-auditor, 1 in agamotto/Penpot).framework/agents/registry.{json,md}+framework/commands/mirrors viacheatheet/generate-{registry,commands}.cjs.Decision matrix (excerpt from
obscura-protocol.md)multi shots.ymlshot-scraper authobscura-worker)obscura serve --port 9222 --stealthTest plan
python3 -m json.tool framework/tools/cli-registry.jsonvalidgo build ./...green inframework/cli/go test ./...green (cmd, config, installer, registry, tui)node framework/cheatheet/generate-registry.cjsregenerates 87 agentsnode framework/cheatheet/generate-commands.cjsregenerates 95 command files./install.sh --with-obscuraon macOS arm64 (binary download +obscura --version)./install.sh --with-obscuraon Linux x86_64cargo install --git ...fallback path (set unsupported platform)visual-auditoragainst a real URL → verifyobscura fetch --evalsucceeds, screenshots still go through shot-scraperobscura-aarch64-macos.tar.gz, etc.) actually exist on the upstream releases page — fall back tocargoif a release is not yet publishedNotes / open points
cargo install --gitautomatically.shot-scraperExternalModule remains opt-in (--with-shot-scraper) per current ulk convention; Obscura follows the same pattern (--with-obscura). The "required" priority in the CLI registry is documentation-level (consumers know it's expected), not enforced at install time.releases/latest/download/obscura-<arch>-<os>.tar.gz) is inferred from the README; reviewers should verify these assets exist before merging, or the install will degrade to the cargo fallback.https://claude.ai/code/session_015HAvQLiLeNyP3LUURaEyeC
Generated by Claude Code
Summary by CodeRabbit
New Features
--with-obscurainstallation option for optional Obscura setup.Documentation